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Introduction 

Project-based learning (PBL) that has authenticity in the pupils’ world 
enables the teaching of science and technology to pupils from a variety of 
backgrounds. PBL has the potential to enable pupils to research, plan, design, 
and reflect on the creation of technological projects (Doppelt, 2000). Imparting 
creative thinking within the design process of pupils’ projects not only requires 
changing the teaching methods and learning environment, but also adopting new 
assessment methods such as student portfolios. Engineering education, which is 
common in Israel, has a unique structure in that it combines practical and 
theoretical knowledge, synthesizes vertical and lateral thinking, and creates a 
rich and flexible learning environment. 

The CTT (Creative Thinking in Technology) program (Barak & Doppelt, 
1999) integrates Co.R.T. Thinking tools (De Bono, 1986) into the technology 
curriculum using the LEGO-Logo learning environment for creating authentic 
projects. The program began in 1994. Pupils study lateral thinking tools in order 
to deal with different alternatives, to consider multiple factors, and to refrain 
from premature judgments on ideas. They use vertical thinking tools in order to 
document their design process and to calculate and to structure programming for 
the control of their projects. Earlier field research by Barak, Waks, & Doppelt 
(2000) showed that pupils prefer a learning environment that emphasizes 
planning and building activities and team projects. Pupils have stated that these 
aspects of a learning environment contribute to creating challenges, curiosity, 
imagination, and success in studying technological subjects (Doppelt & Barak, 
2002). As the CTT program evolved, a Creative Thinking Scale (CTS) was 
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developed in order to assess pupils’ portfolios (Barak & Doppelt, 2000). The 
study reported herein extends research conducted on the CTT program and 
proposes a Creative Design Process (CDP) that can be assessed using the CTS. 

Theoretical Background 

The theoretical background of this study cuts across three areas of inquiry: 
engineering education, the infusion of creative thinking into the design process, 
and the assessment of project-based learning. 

Engineering Education: A MECHATRONICS Context 

De Vries (1996) claimed that we should help pupils integrate knowledge 
from science and other disciplines into the design processes. It is evident that 
there is a role for science education and that science education remains a crucial 
part of general education, even where technology education has gone beyond 
the ‘‘‘technology is applied science” paradigm. Technology education is an 
equally valuable subject to science education, and both subjects should be 
taught (Gardner, 1997). 

Engineering education in Israel is part of the comprehensive high school 
curriculum. At the end of junior high school, pupils have to choose one or more 
areas as a “major,” such as the Sciences, Humanities or Technology. Within 
Technology is Engineering. The Engineering curriculum for high school in 
Israel contains several core subjects that are related to physics and mathematics, 
such as civil engineering, computers and electronics, and mechanics and control 
systems. MECHATRONICS is a new sub-major in the mechanics and control 
area. Pupils study MECHATRONICS for three years, 10th - 12th grades. They 
study physics, system control, mechanics, and programming for five hours per 
week, with a total time devoted to all classes of 20 hours per week. This 
syllabus is about half of their weekly schedule. 

The MECHATRONICS syllabus has been implemented in nine schools 
since 2000/2001. In 2003 there were fifteen schools that chose to advise their 
pupils to take this syllabus. The MECHATRONICS curriculum suggests that 
educators create rich learning environments filled with real world applications. 
The assessment processes of learning outcomes in a rich learning environment 
have had an important impact on the learning process (Doppelt & Barak, 2002). 
In addition, the perspective of pupils on the most influential characteristics of 
the learning environment is important for the teaching-learning process and for 
designing the learning environment (Doppelt, 2004). This study suggests a way 
of infusing creative thinking into the design process during project-based 
learning (PBL). 

Infusing Creative Thinking into the Design Process 

The design process is similar to the process of problem solving. Many 
general structures for design processes appear in the literature. A generalized 
approach to structuring the design process might include six stages: defining the 
problem and identifying the need, collecting information, introducing alternative 
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solutions, choosing the optimal solution, designing and constructing a prototype, 
and evaluating and correcting the process. The design process might also 
include four thinking levels: 

1. Understanding existing systems 

2. Systematic and functional understanding of engineering systems 

3. Applying set procedures for analysis and synthesis 

4. A controlled design process that assures that the above steps are 
applied by the students. 


(Mioduser, 1998) 


Teaching technology using a generalized design process has been criticized 
by researchers who have claimed that it is difficult for pupils, and even for 
teachers, to learn how to use it (McCormick & Murphy, 1994). This is not 
surprising given that different technology disciplines go about designing in 
different ways (Hill & Anning, 2001). In order to avoid the pitfalls of a 
generalized design process, teachers should assist pupils to integrate the 
knowledge of the various disciplines and learn the applicable standards and 
rules, as well as the underlying scientific principles and economic concepts (De 
Vries, 1996). This article will describe a design process that applies these 
suggestions and is sensitive to the authenticity of the pupils’ projects. This 
design process was based initially on the PISCO framework (De Bono, 1986). 
PISCO stands for: Purpose, Input, Solutions, Choice and Operations. Infusing 
the teaching of thinking skills into a specific disciplinary course may provide a 
rich learning environment that will contribute not only to the development of 
thinking skills but also to a better understanding of the discipline under study 
(Glaser, 1993; Ennis, 1989; Zohar & Tamir, 1993). 

De Bono (1986) differentiates between two types of thinking: lateral 
thinking, which refers to discovering new directions of thinking in the quest for 
a wealth of ideas, and vertical thinking, which deals with the development of 
ideas and checking them against objective criteria. Lateral thinking and vertical 
thinking are quite different processes. It is not a matter of one process being 
more effective than the other, as both are necessary. Rather, in order to be able 
to use both effectively, one must appreciate their differences. Lateral thinking is 
a central, but not singular, component of creative thinking. Waks (1997) pointed 
out that during work on a technological project, lateral thinking initiates the 
learning process while pupils seek alternatives and examine different solutions. 
Vertical thinking is essential at the stage of choosing a solution and developing 
it. Vertical thinking and lateral thinking complement each other, and both are 
the essential elements of creative thinking (De Bono, 1986). 

Assessing Creative Thinking in PBL 

Technological systems that are controlled by a computer provide a rich 
learning environment and expose the learner to a variety of representations and 
configurations, such as realistic models, simulations, mathematical models, 
algorithms, graphics, and animations. One of the better-known examples of such 
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a rich computer-based learning environment is the LEGO-Logo system (Doppelt 
& Armon, 1999; Jarvinen, 1998). Resnick and Ocko (1991) stated that this 
learning environment puts children in control because they formulate their own 
designs and experiments and work on projects that they care about personally 
instead of reproducing someone else's experiment. 

Project based learning in technology encourages pupils to work in teams 
(Barak & Maymon, 1998; Denton, 1994). In this way, pupils combine “hands- 
on” activities with what Papert (1980) has termed “heads-in” activities. Project- 
based learning (PBL) could be used as a tool to develop pupils’ competencies 
by working on integrated projects (Barlex, 2002). Project-based learning 
through authentic issues taken from the pupils’ world enables pupils from 
various backgrounds to study science and technology in a way that makes sense 
to them (Seiler, Tobin & Sokolic, 2001). An authentic project deals with real 
life situations and, by definition, is integrative in nature. This approach has been 
implemented throughout the world and past researchers have shown interesting 
findings regarding the opportunities afforded by PBL (Barak & Doppelt, 1999, 
2000; Barak, Eisenberg & Harel, 1995; Barlex, 1994; Doppelt, 2003; Doppelt & 
Barak, 2002; Hill, 1998; Resnick & Ocko, 1991). One of the key elements for 
success in PBL is engaging the pupils in the assessment process. 

Imparting creative thinking within science and technology education not 
only requires changing the teaching methods and learning environment, but also 
adopting new assessment methods, such as portfolio assessment, which is based 
on records of pupils’ activities. The portfolio might consist of such items as 
written material, computer files, audio and video media, sketches, drawings, 
models, and pictures. The portfolio reflects what pupils have learned, how they 
question, analyze, synthesize, solve problems, and create new ideas, and how 
then design and build useful products or systems. The portfolio also shows how 
pupils interact intellectually, emotionally, and socially with others (Collins, 

1991; Wolf, 1989). 

Barak and Doppelt (1999) have shown that pupils were able to cope with 
complex problems and they developed solutions that depended on creativity, 
supporting the notion that they applied lateral and vertical thinking. Pupils 
created portfolios containing documentation of their creative thinking and the 
learning processes in which they were engaged. Over a period of several years, 
each class developed criteria for assessing their portfolios. On the basis of these 
experiences, a Creative Thinking Scale (CTS) was developed and was formally 
applied to the assessment of pupils’ portfolios (Barak and Doppelt, 2000). 

The suggested assessment scale of creative thinking can help educators 
strive for a gradual development of higher-order thinking skills in two main 
areas. The first is the choice of a project topic that that pupils make. This 
includes dimensions of complexity, originality, and creativity on the one hand 
and the extent of mathematical, logical, and scientific thinking on the other. The 
second area considers the thinking and learning processes applied as the pupils 
developed their project and includes problem solving, teamwork, and reflective 
thinking. Thus, learning through the designing and implementation of 
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technology projects based on portfolio assessment and directed towards a 
systematic development of vertical and lateral thinking may promote teaching 
and learning that assist the successful integration of the pupil into the dynamic 
and changing world outside of the school once they graduate. 

Background and Precursors 


Intervention Program 

The intervention program included in-service training of teachers for a 
duration of 112 hours. This consisted of twenty-eight workshops of four hours 
per meeting during the school year. The program was aimed at introducing three 
main topics: the tutoring process, the Creative Design Process (CDP), and 
assessment using the Creative Thinking Scale (CTS) criteria. Eighteen teachers 
from nine schools who had taught MECHATRONICS since 2000 were involved 
in the project. 

Six Stages of the Creative Design Process (CDP) 

The creative thinking framework was originally developed by De Bono 
(1986). He based this framework on five thinking steps that he named “PISCO” 

- Purpose, Input, Solutions, Choice and Operations. The Creative Design 
Process (CDP), which is presented in this article, adopts this creative thinking 
framework and extends it to project-based learning (PBL) in technology 
education. 

The creative thinking tools that are suggested in the CDP are part of the 
Co.R.T thinking program (De Bono, 1986). They include P.M.I (Plus, Minus, 
Interesting), C.F.A (Consider All Factor), Rules, C&S (Consequence and 
Sequel), O.P.V (Other People’s View) and F.I.P (First Important Priorities). 

First stage: Design Purpose. The first step in the design process is defining the 
design problem. The pupils need to set the design goals. These goals must fit the 
definitions of the problem. The achievement of these goals will be under the 
restrictions that the designer has set forth and include budget, availability, 
equipment and tools, schedule, and so on. Three steps are recommended to the 
pupils for the documentation of this first stage: 

1 . The problem and the need - Pupils describe the reasons that motivated them 
to choose their project. They also define the problem and define the needs 
that their solution will address. 

2. The target clientele and restrictions - Pupils describe the target clientele 
and define the restrictions that they must take into consideration. The Rules 
thinking tool can help you consider rules, standards, and other restrictions. 

3. The design goals - The pupils define the necessary demands they expect to 
be met by the system. 

Second stage: Field of Inquiry. The second step in the design process is to 
define the field of inquiry in which their problem resides. This is founded on the 
problem definitions and goals from the first step. Pupils must research and 
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analyze existing systems that are similar to the one they are developing. Pupils 
need to organize the documentation of their inquiry. This includes the following 
main areas: 

1 . Information Sources - Books, professional magazines, manufacturers’ 
catalogs, and Internet sites. 

2. Identification of Engineering, Scientific, and Societal Aspects - Engineering 
concepts, scientific concepts, societal and environmental aspects, cultural 
values, and potential issues and dilemmas. 

3. Organization of the Information and its Assessment - Arrangement of the 
information according to the goals and restrictions of the problem. Pupils 
need to summarize the information gathered so that the design problem and 
approach are informed by it. Pupils should express their opinions regarding 
the match of the information they have gathered to their design problem. 
They must also provide a rationale for why their evolving design is a better 
alternative to systems that already exist. 

Third stage: Solution Alternatives. The third step in the Creative Design Process 
involves the consideration of alternative solutions to the design problem. This is 
a lateral thinking stage and includes three components: Ideas, Factors, and 
Opinions (of other people). Pupils need to be educated so that they feel freedom 
in their thinking and are not discouraged by the judgments that other pupils and 
their friends might make. This strategy will increase the likelihood that 
numerous design possibilities will be considered, with the intent of inspiring 
creativity and arriving at an idea that no one else has developed. There are no 
bad ideas at this stage. The pupils are presented the following suggestions and 
guidelines: 

1 . Ideas Documentation - The use of the P.M.I (Plus, Minus, Interesting) 
thinking tool is recommended to the pupils so that they consider as many 
ideas as possible and formally evaluate them, considering all aspects. The 
aspects of the ideas are rated as positive, negative, or interesting, meaning 
that they have promise but more investigation is needed. 

2. Consider All Factors (C.A.F) - Pupils are asked to write down all the 
factors related to the system they are designing. They must consider the 
perspectives of the consumer, the designer, the manufacturer and the 
marketer. 

3. Consequence and Sequel (C&S) - Pupils must consider and document all of 
the consequences of each of their ideas on such elements as the 
environment, society, and the individual. Both short term and long term 
consequences must be considered. 

4. Other People’s View (O.P. V): - Pupils must seek the opinions of others 
about their ideas and they must document what they find. 

Fourth stage: Choosing the Preferred Solution. - The fourth step in the design 
process is choosing the preferred solution. The choice is made from the various 
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ideas that were documented in the third stage. The solution chosen will possibly 
fulfill the following criteria: 

1. Have the largest amount of plus and/or interesting points and the least 
minus points 

2. Considers as many factors and viewpoints as possible. 

3. Is adequate in both the short and long term. 

4. Appears to be a good solution in the minds of others. 

5. Meets all the requirements set forth in the definition of the problem. 

Table 1 is designed to assist pupils in choosing the best solution. 

Table 1 


An evaluation scheme for evaluation pupil ideas 



Very 

Weak 

(1) 

Weak 

(2) 

Average 

(3) 

Good 

(4) 

Very 

Good 

(5) 

Positive Points 






Interesting Points 






Factors Involved 






Viewpoints 






Short/Long Term 






Other People View 






Necessary Demands 






Desirable Demands 







The use of the F.I.P (First Important Priority) thinking tool is recommended to 
pupils in order to help them set priorities. This thinking tool could assist in 
choosing the optimal solution. 

Fifth stage: Operation Steps. The fifth step in the design process is planning the 
operational ways to implement the chosen solution. Planning must be consider 
the following points: 

• Sometimes the chosen solution is a complex system. Dividing it into sub¬ 
systems may assist in defining the steps that are needed to develop the 
solution. 

• In many cases choosing the ideal materials, parts, and mechanisms are a 
central part of the design process. 

• The sketches and drawings (computer-generated as well as traditional) are 
important to the presentation of a design. 

• Choosing machines, tools, and manufacturing processes are necessary steps 
to creating a prototype. 

• Planning how to make the prototype is critical to success. Planning activities 
include developing a timeline of tasks and making sure the necessary 
materials, parts, machines, human resources, and so forth are available and 
will be available when needed. 
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Six stage: Evaluation. The last step is to evaluate the overall process and 
product. This is a summative evaluation and relates back to the formative 
evaluation steps that were done as the idea was developing. In this final stage 
the pupils need to document: 

• What difficulties were encountered and what methods were used to 
overcome them? 

• Does the system actually provide a viable solution to the problem? 

• Does the prototype fulfill all its performance requirements? 

• In what way could the prototype be improved? 

• What are the implications for further development? 

The Creative Thinking Scale 

In this study the Creative Thinking Scale (CTS) was used to assess the 
suggested Creative Design Process (CDP). The CTS was introduced to the 
teachers during the workshops mentioned earlier. It consists of the four thinking 
levels that De Bono (1996) defined: 

1. Awareness of Thinking 

The first level deals with developing an awareness that thinking is a skill 
that can be developed. Pupils are taught how to prepare to engage in 
thinking about something, how to conduct inquiry, and how to listen to and 
evaluate the opinions of others. 

2. Observation of Thinking 

The second level deals with observing the consequences of action and 
choice, considering other people’s views, and comparing alternatives. 

3. Thinking Strategy’ 

The third level deals with the directed use of some thinking tools, 
organizing thinking as a sequence of steps, and using thinking to define 
goals. 

4. Reflection upon Thinking 

The fourth level deals with a systematic use of thinking tools, clear 
awareness of the need for reflective thinking, self-evaluation of thinking, 
designing thinking tasks, and methods to implement these tasks. 

The CTS evaluates the pupils’ portfolios across two dimensions. The first 
considers the design, construction, and evaluation of the product or system. 
Evidence of lateral thinking, including originality, authenticity, usefulness, and 
unique design is sought. Likewise, evidence of vertical thinking is also sought 
and includes functionality, reliability, accuracy, geometric structure, and the 
application of scientific principles. 

The second dimension considers the processes of learning and includes 
thinking, problem solving, and teamwork. Evidence is sought of individual and 
group efforts in problem solving, collaborative decision-making, and leadership. 
Table 2 presents the CTS, the development of which was detailed elsewhere 
(Barak & Doppelt, 2000). 
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Table 2 

The Creative Thinking Scale (CTS) 


Achievement Levels 

Portfolio’s Components | 

Design, construction, and 
evaluation of the system 
or product 

Learning, thinking, and 

problem-solving 

activities 

Level 1: Awareness 

The awareness to consider 
thinking as a skill which 
can be developed; prepare 
to think about something; 
prepare to inquire; prepare 
to listen to other people 
opinions. 

Standard diagram of a 
system or product taken 
from available literature. 
Basic explanation of the 
model and its construction. 
Description of the model 
by means of pictures or 
sketches. 

An example of solving a 
simple problem in 
planning and construction. 
Division of tasks among 
the team members. 

A few examples of using 
lateral and vertical 
thinking tools. 

Level 2: Observation 

The observation of 
consequences of action 
and choice; consider the 
views of others; compare 
alternatives. 

Original schematic 
diagram of system or 
product designed by the 
pupil. Detailed drawings of 
the model. 

Specification of planning 
and construction stages 
including calculations, 
specifications or computer 
programs. 

Justified examples of 
choices among a number 
of alternatives. 

Information exchange and 
reciprocal help in the team. 
Various examples of using 
thinking tools. 

Level 3: Strategy 

The directed use of some 
thinking tools; organizing 
the thinking as a sequence 
of steps; using thinking to 
define goals. 

Original system functional 
block diagrams, structural 
tree or flow chart. 
Description of a number of 
iterations in the planning 
and construction of the 
model. 

Comparison among 
possible models and 
choosing from them. 

Examples of the 
contribution of individuals 
and teamwork to solving 
complex problems. 

Evidence of the planned 
use of the thinking tools, 
open-mindedness, and 
postponing decision 
making (lateral thinking); 
setting priorities, goals and 
criteria (vertical thinking). 

Level 4: Reflection 

A systematic use of 
thinking tools; clear 
awareness of reflective 
thinking; self-evaluation of 
thinking; designing 
thinking tasks and methods 
to implement these tasks. 

Examination of the final 
product’s features, 
compared to the set goals. 
Conclusions on successes 
or difficulties during the 
development process. 
Suggestions for 
improvement in the 
planning and construction 
process. 

Conclusions drawn from 
the influence of the team's 
collaboration on the 
completion of the project. 
Pupils’ view on the 
influence of the team’s 
functioning on thinking 
and learning processes. 
Assessment of the selected 
solution compared to the 
goals. 
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Method 

This study aimed first to investigate the way pupils design their projects. 
The second aim was to describe the assessment of Project-Based Learning 
(PBL). The third aim was to explore the ways teachers set goals for their pupils 
according to the CDP and the CTS. A qualitative approach was used in order to 
foster collaboration with teachers and to gain close interaction with pupils, 
teachers, principals, and supervisors from the Israeli Ministry of Education. 

In 2003 there were nine schools that offered MECHATRONICS as a major 
during the 10 th -12 th grades (16-18 years old). Eighteen teachers were involved in 
delivering the MECHATRONICS courses. Approximately 180 pupils have 
graduated from high schools with MECHATRONICS as their major. Pupils’ 
projects are examined through a matriculation examination at the end of the 
12th grade. In these examinations a supervisor arrives at each school and the 
pupils are required to present a portfolio of their design process and the final 
product or system that resulted. Team projects are assessed the same as single 
projects. The examination supervisor asks questions of each of the pupils in the 
team. 

A national contest was organized for all the Israeli pupils in the 
MECHATRONICS programs. All the pupils knew at the outset of the contest 
that they would be required to design, construct, and to program a system that 
would assist humans. Criteria for assessing the projects were developed with the 
teachers. The teachers were instructed to introduce the criteria to their pupils six 
months prior to the contest. The assessment process during the contest itself 
focused on evaluating the design process and resultant products based upon a 
presentation made by the pupils. 

Agreement was reached among the teachers on the criteria for assessing the 
pupils’ work during a meeting six months before the contest. These criteria were 
validated in advance of this meeting through the agreement of five senior 
teachers from schools that were not participating in the contest and five 
researchers from the Technion - Israel Institute of Technology. These ten 
individuals were invited at the contest to serve voluntarily as members of the 
assessment committee. 

Participants 

In all nine schools the teachers assessed the projects using the developed 
criteria. Each school could send three teams to the contest. Fifty-four pupils 
were chosen by their teachers to participate in an Israeli national 
MECHATRONICS contest, which was held at the Technion. The pupils were in 
the second semester of their 12th grade and they had been learning 
MECHATRONICS according to the syllabus that was described earlier. The 
teachers of these pupils, eighteen from the nine participating schools, applied 
PBL in their classes during the 12th grade. 
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Data Collection and Analysis 

As mentioned earlier, the CDP and the CTS had been introduced to the 
teachers during a workshop. The author actively participated in the workshop. 
During the workshop, criteria for assessing PBL were agreed upon among the 
teachers. These criteria were used to assess the pupils’ projects in the national 
contest. Six month prior to the contest, the teachers and pupils became familiar 
with the assessment criteria. The pupils were required to present their projects 
during the contest. By way of these presentations, the researcher was able to 
investigate the implementation of the CDP and the impact of the CTS upon 
pupils’ projects. 


Findings 

The findings are presented in three categories. First, projects are briefly 
described, showing the variety and authenticity of the pupils’ work. Second, a 
representative example of the assessment process is presented. Finally, the 
judges’ assessment of the MECHATRONICS projects in the national 
competition is presented. 

The Projects 

Fifty-four pupils were eligible to enter the national MECFIATRONICS 
contest, representing 18 projects. Of these, three teams were unable to bring 
their projects to the final level needed to actually compete in the contest. In the 
end, forty five pupils representing 16 projects were in the competition. Only one 
project was done by a student working alone. 

The Judging Process 

The pupils presented their projects to an assessment committee consisting 
of teachers from the participating school, senior teachers from schools that were 
not participating in the contest, and researchers from the Technion - Israel 
Institute of Technology. Procedures were used to assure that a given project was 
not judged by a teacher of the pupils who produced it. 

There were two stages to the assessment process. In the first stage, the 
projects were presented to teams of three judges. Approximately three projects 
were evaluated by each team of judges. From this procedure, six projects moved 
on to the second, or final, stage of the evaluation process. The descriptions in 
Table 3 show that the projects were authentic and varied. 

Table 4 presents a representative project, which demonstrates the 
assessment process. This example is project No. 7 from the final assessment 
presented in Table 3. These findings show a high-level of agreement among the 
independent assessments of each criterion as scored by the judges. The scale for 
the scoring was 10 percent for the first and the last criteria, and 20 percentages 
for each of the other criteria. 
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Table 3 

MECHATRONICS Projects 


Pupils 


Project 


on 

No. 

Project Description 

Team 

1 

Automated control system for a hoisting machine 

4 

2 

Automated, multi-player basketball game 

3 

3 

Simulator for jogging 

2 

4 

Automated system for changing and playing compact 
discs 

1 

5 

Computerized Scanner 

2 

6 

System for coaching Ping-Pong players 

6 

7 

Simulation of riding a bicycle 

2 

8 

Automated system for finding and collecting tennis balls 

2 

9 

Computerized system for playing chess. 

3 

10 

Computer controlled model of a detention facility 

2 

11 

Computer controlled system for coaching boxers 

5 

12 

Automated system for replacing wheels 

3 

13 

System to assist pupils in learning about computer 
control 

2 

14 

System for coaching tennis players 

3 

15 

Automated system for identifying and neutralizing 
bombs 

3 

16 

Computer controlled system for pumping water 

2 

Total 


45 

Table 4 



Independent judgment according to agreed criteria 


Criteria 

Judge Judge Judge 

No. 1 No. 2 No. 3 

Mean 


Presenting the needs and the 
system’s goal 

9 

8 

10 

9.0 

Presenting alternative and 
creative solutions 

19 

15 

18 

17.3 

Analyzing the chosen system 

18 

15 

18 

17.0 

Performance of a working 
controlled prototype 

12 

13 

12 

12.3 

Sophistication of the control 
program 

20 

17 

18 

18.3 

Presentation of design stages 

8 

7 

8 

7.7 

Total 

86 

75 

84 

81.7 
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Judge 1 was a senior teacher who had not participated in the intervention 
but was familiar with the criteria. The correlation between judges 1 & 2 was 
0.957; between judges 1 & 3 it was 0.990 and between judges 2 & 3 it was 
0.944. Similarly, high correlations were found among all of the other 
assessments, which were scored by the other judging teams. The assessments at 
the same time and there was no interaction between one team and the other 
teams. This shows that the judges have a shared perception of the criteria. 

The Final Assessment of the Projects 

Table 5 presents the findings from the final assessment process. In the final 
stage the pupils presented the six projects, shown in bold in the table, to the 
committee in a large auditorium. In the auditorium 160 guests watched the final 
contest and included families, teachers, school principals, supervisors from the 
Ministry of Education, and researchers from the Technion. The assessment 
committee watched each presentation and assessed the six projects according to 

Table 5 

Final scoring of each participating project _ 


Rank order of 
projects 

Presents the pu 
of the system 

Presents alterni 
and ct 

Analysis of the 
chosen system 

A working, 
controlled proti 

Complexity of t 
control prograr 

Presents the dei 
stages 

*<Z5 

& 

© 

Q- 

O 

o 

t-H 

s 

© 

C/5 

1 

10.0 

17.7 

20.0 

20.0 

19.7 

8.7 

96.0 

2 

10.0 

17.7 

19.7 
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the developed criteria, as mentioned previously. The six projects were an 
automated controlled system for lifting a hoister weight, an automated 
multiplayer basketball game, a mini football game, a jogging simulator, an 
automated system for changing and playing compact discs, a computerized 
scanner, and a system for coaching ping-pong players. 

Discussion 

Seven years of experience in implementing the CTS in order to assess 
pupils’ projects demonstrates how this methodological assessment can help 
educators develop and evaluate learning assignments aimed at fostering creative 
thinking in technology (Doppelt & Barak, 2002). Through the CDP and 
systematic reflection on it, pupils can develop awareness of their internal 
thinking processes and document them. The purpose is not to educate pupils to 
design according to some generalized procedure, external to them, for 
constructing their ideas, solutions, and products (De Vries, 1996). This is 
counter to creativity. Rather, it is an educational goal to teach them to document 
their thinking properly and thereby enable them to reflect on their creations and 
how they developed them. Research was recently conducted in this area by 
Doppelt, Mehalik, and Schunn (2005). 

Pupils are expected to internalize their adaptation of the design process, to 
use it in their own way, to apply it to new situations, and to demonstrate general 
patterns of lateral and vertical thinking in their technology projects. No less 
important is fostering pupils’ meta-cognition, or ‘thinking of thinking’. The way 
pupils commence, progress, and complete their project demonstrates that 
creative thinking in technology is a combination of vertical and lateral thinking 
(Waks, 1997; Barak & Doppelt, 2000). 

In addition, the projects reported herein show that pupils in high school can 
create, design, implement, control, and document authentic, real-life projects 
instead of solving well-defined problems prescribed by the teacher. In fact, the 
criticism of current engineering education is that there is an overemphasis on 
solving well-defined, closed-ended problems (NSPE, 1992). Furthermore, 
pupils have proven through their projects that they are capable of dealing with 
the “large definition of DESIGN” - that the DESIGN activity does, in fact, 
encompass the entire process of planning, designing, constructing, and 
managing the development of a product (De Vries, 1993; Hill, 1998). 

The CTS has enabled teachers and researchers to set goals for the pupils 
(and for the teachers) during the PBL. The consistency of the judges’ scores and 
the successful application of the criteria developed by the teachers strengthened 
their validity. The findings of the assessment process indicate that the CDP and 
the CTS are useful and can be implemented by teachers who have participated 
in a suitable in-service training. The assessment of technology education can 
serve as a highly integrative element in technology education that allows pupils 
to combine and integrate various knowledge and skills (De Vries, 1997). 
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Conclusions 

The in-service teacher workshop assisted the teachers in discussing tutoring 
issues, design stages, and assessment criteria. The independent judging showed 
a high-level of agreement among the judges. These assessments were similar to 
the researchers’ and senior teachers’ assessment in the final stage of the contest. 
The researchers and senior teachers had not taken an active part in the tutoring 
process during the school year. They were familiar, however, with the criteria 
and had agreed upon them. This model can be adopted in other relevant issues 
regarding collaboration between field practice and academic research. 

This article introduced the Creative Design Process (CDP) and the creative 
thinking scale (CTS). The CDP is aimed at assisting pupils in documenting the 
design process. The CTS could be used as a guideline for teachers during their 
tutoring and for pupils during the development of creative solutions to 
problems. The findings showed that pupils learned to document their design 
process according to the CDP. Finally, teachers turned into better tutors after 
they became familiar with the CTS. 

The implementation of the CTS concerning the outcomes of the CDP has 
important consequences for the professional development of teachers and for the 
development of pupils’ skills. Teachers can use the CTS as the goal of their 
teaching. If the CTS is introduced together with the CDP to pupils, they can 
develop their competencies according to various learning styles. The 
methodological assessment used in this study during the intervention program 
with the teacher and during the contest with teachers, senior teachers and 
researchers can be used in other science and technology domains. The contest 
was also found to be a useful instrument to enhance collaboration between 
researchers and teachers and among schools. This research could add a relevant 
body of knowledge to the assessment of technology education. 
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